Augmented-reality systems often portray digitally enhanced images or other scenes with computer-simulated objects. To portray such scenes, an augmented-reality system sometimes renders both real objects and computer-simulated objects with shading and other lighting conditions. Many augmented-reality systems attempt to seamlessly render virtual objects composited with objects from the real world. To achieve convincing composites, an augmented reality system must illuminate virtual objects with consistent lighting matching a physical scene. Because the real-world is constantly changing (e.g., objects move, lighting changes), augmented-reality systems that pre-capture lighting conditions often cannot adjust lighting conditions to reflect real-world changes.
Despite advances in estimating lighting conditions for digitally enhanced scenes, some technical limitations still impede conventional augmented-reality systems from realistically portraying lighting conditions on computing devices. Such limitations include altering lighting conditions when a digitally enhanced scene changes, quickly rendering or adjusting lighting conditions in real (or near-real) time, and faithfully capturing variation of lighting throughout a scene. These limitations are exasperated in three-dimensional scenes, where each location at a given moment can receive a different amount of light from a full 360-degree range of directions. Both the directional dependence and the variation of light across the scene play a critical role when attempting to faithfully and convincingly render synthetic objects into the scene.
For example, some conventional augmented-reality systems cannot realistically portray lighting conditions for a computer-simulated object in real (or near-real) time. In some cases, conventional augmented-reality systems use an ambient-light model (i.e., only a single constant term with no directional information) to estimate the light received by an object from its environment. For example, conventional augmented-reality systems often use simple heuristics to create lighting conditions, such as by relying on mean-brightness values for pixels of (or around) an object to create lighting conditions in an ambient-light model. Such an approximation does not capture the directional variation of lighting and can fail to produce a reasonable ambient-lighting approximation under many conditions—resulting in unrealistic and unnatural lighting. Such lighting makes computer-simulated objects appear unrealistic or out of place in a digitally enhanced scene. For instance, in some cases, conventional systems cannot accurately portray lighting on objects when light for a computer-simulated object comes from outside the perspective (or point of view of) shown in a digitally enhanced image.
In addition to challenges to portraying realistic lighting, in some cases, conventional augmented-reality systems cannot flexibly adjust or change lighting conditions for a particular computer-simulated object in a scene. For instance, some augmented-reality systems determine lighting conditions for a digitally enhanced image as a collective set of objects or as an image as a whole—instead of lighting conditions for particular objects or locations within the digitally enhanced image. Because such lighting conditions generally apply to a set of objects or an entire image, conventional systems either cannot adjust lighting conditions for a particular object or can only do so by redetermining lighting conditions for the entire digitally enhanced image in an inefficient use of computing resources.
Independent of technical limitations affecting the realism or flexibility of lighting in augmented reality, conventional augmented-reality systems sometimes cannot expeditiously estimate lighting conditions for objects within a digitally enhanced scene. For instance, some conventional augmented-reality systems receive user input defining baseline parameters, such as image geometry or material properties, and estimate parametric light for a digitally enhanced scene based on the baseline parameters. While some conventional systems can apply such user-defined parameters to accurately estimate lighting conditions, such systems can neither quickly estimate parametric lighting nor apply an image-geometry-specific lighting model to other scenes with differing light sources and geometry.